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Abstract: Based on our previous work we propose to track multiple ground targets 
with GMTI (Ground Moving Target Indicator) sensors as well as with imagery sensors. 

The scope of this paper is to fuse the attribute type information given by heterogeneous 
sensors with DSmT (Dezert Smarandache Theory) and to introduce the type results in 
the tracking process to improve its performances. 



1 Introduction 

Data fusion for ground battlefield surveillance is more and more strategic in order to cre- 
ate the situational assessment or improve the precision of fire control system. For this, we 
develop new ground target tracking algorithms adapted to GMTI (Ground Moving Target 
Indicator) sensors. In fact, GMTI sensors are able to cover a large surveillance area during 
few hours or more if several sensors evolve on the same operational theatre. Several ref- 
erences exist for the MGT (Multiple Ground Tracking) with GMTI sensors [?, 8] whose 
fuse contextual informations with MTI reports. The main results are the improvement of 
the track precision and track continuity. Our algorithm [6] is built with several reflexions 
inspired of this literature. The proposed VS-IMMC (Variable Structure Interacting Mul- 
tiple Models) filter is extended in a multiple target context and integrated in a SB-MHT 
(Structured Branching - Multiple Hypotheses Tracking). 

One way to enhance data associations is to fused data obtained by several sensors. The 
most easily approach is to consider the centralized fusion between two or more GMTI 
sensors. Another way is to introduce heterogeneous sensors in the centralized architecture 
in order to improve the data associations (by using the reports in location and its classifi- 
cation attribute) and palliate the poor GMTI sensor classification. In our previous works 
[6], the classification information of the MTI segments and IMINT segments (IMagery 
INTelligence) has been introduced in the target tracking process. The idea was to main- 
tain aside each target track a set of ID hypotheses. Their committed belief are revised in 
real time with the classifier decision through a very recent and efficient fusion rule called 
proportional conflict redistribution (PCR). 

In this paper, in addition to the measurement location fusion, we illustrate on a complex 
scenario our approach to fuse MTI classification type with image classification type asso- 
ciated to each report. 
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2 Motion & observation models 

2.1 Constrained motion model 

The target state x(fc) at the current time tp- is defined in a local horizontal plane (O, X , Y) 
of a Topographic Coordinate Frame denoted TCF. The target state on the road segment s 
is defined by x s (/c) where the target position (x s (k),y s (k)) belongs to the road segment s 
and the corresponding heading (x s (k) 1 y s (k)) is in its direction. The event that the target 
is on road segment s is noted e s (k) = {x(fc) £ s}. Given the event e s (k) and according 
to a motion model M t , the estimation of the target state can be improved by considering 
the road segment s. The constrained motion model M? is build in such a way that the 
predicted state is on the road segment s and the gaussian noise is defined under the road 
segment constaint [6]. After the state estimation obtained by a Kalman filter, the estimated 
state is then projected according to the road constraint e s (k). This process is detailed in 
[ 6 ]. 



2.2 GMTI measurement model 

According to the NATO GMTI format [5], the MTI reports received at the fusion station 
are expressed in the WGS84 coordinates system. The MTI reports must be converted in the 
TCF. A MTI measurement 2 at the current time tk is given in the TCF. Each MTI report is 
characterized both with the location and velocity information (range radial velocity) and 
also with the attribute information and its probability that it is correct. We denote Cmti 
the frame of discernment on target ID based on MTI data. Cmti is assumed to be constant 
over the time and consists in a finite set of exhaustive and exclusive elements representing 
the possible states of the target classification. In this paper, we consider only 3 elements 
in Cmti defined as Cmti = {Tracked vehicle , Wheeled vehicle , Rotary wing aircraft}. 

We consider also the probabilities P{c(k)} (Vc(fc) £ Cmti) as input parameters of our 
tracking systems characterizing the global performances of the classifier. The vector of 
probabilities [P(ci) P(c 2 ) -P(c3)] represents the diagonal of the confusion matrix of the 
classification algorithm assumed to be used. Let z* MTI (k) the extended MTI measure- 
ments including both kinematic part and attribute part expressed by the herein formula: 

z MT/( fc ) - {z MTi{k),c(k),P{c(k)}} (1) 



2.3 IMINT motion model 

For the imagery intelligence (IMINT), we consider two sensor types : a video EO/IR sen- 
sor carried by a Unmanned Aerial Vehicle (UAV) and a EO sensor fixed on a Unattended 
Ground Sensor (UGS). We assume that the IMINT reports z V ideo{k) at the current time tk 
are expressed in the reference frame ((), X , Y) and give a location information and type 
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target. We assume that the video information given by both sensor types are processed by 
their own ground stations and that the system provides the video reports of target detections 
with their classification attributes. For the last point, a human operator selects targets on 
a movie frame and is able to choose its attribute with a HMI (Human Machine Interface). 
Based on the military symbology called 2525C [3], we build the frame of discernment for 
an EO/IR source denoted C V ideo ■ Each video report is associated to the attribute infor- 
mation c(k)(y c(k) £ C V ideo) with its probability P{c(fc)} that it is correct. As Cmti, 
C V ideo is assumed to be constant over the time and consists in a finite set of exhaustive and 
exclusive elements representing the possible states of the target classification. 

Let z v ideo {k) be the extended video measurements including both kinematic part and at- 
tribute part expressed by the following formula (Vc(fc) £ C video)' 

Kideo ( k ) - { z video(k),c(k),P{c(k)}} (2) 

The attribute type of the image sensors belongs to a different and better classification than 
the MTI sensors. 



2.4 Taxonomy 

In our work, the symbology 2525C [3] is used to describe the links between the different 
classification sets Cmti and C V i de o ■ Figure 1 represents a short part of the 2525C used 
in this paper. The red elements underlined in italic style are the atomic elements of our 
taxonomy. Each element of both sets can be placed in 1. For example, the “wheeled ve- 
hicle” of the set Cmti is placed at the level “Armoured — > Wheeled” or the “Volkswagen 
Touareg” given by the video is placed at the levels “Armoured — > Wheeled— > Medium” 
and “Civilan Vehicle — - Jeep — > Medium”. 



3 Tracking with road constraints 

3.1 VS IMM with a road network 

The IMM is an algorithm for combining state estimates arising from multiple filter models 
to get a better global state estimate when the target is under maneuvers. In section 2.1, a 
constrained motion model i to a road segment s, noted M*(fc), was defined. We extend the 
segment constraint to the different dynamic models (among a set of r + 1 motion models) 
that a target can follow. The model indexed by r = 0 is the stop model. It is evident that 
when the target moves from one segment to the next, the set of dynamic models changes 
according to the road network configuration. The steps of the IMM under road segment s 
constraint are the same as for the classical IMM as described in [1], 

In real applications, the predicted state could also appear onto another road segment, be- 
cause of a road turn for example, and we need to introduce new constrained motion models. 
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Figure 1 : 2525C (light version). 



In such case, we activate the most probable road segments sets depending on the local pre- 
dicted statelocation of the track l' kJ [6]. We consider r + 1 oriented graphs which depend 
on the road network topology. For each graph i, i = 0, 1, . . . , r, each node is a constrained 
motion model M* . The nodes are connected to each other according to the road network 
configuration and one has a finite set of r + 1 motion models constrained to a road section. 
The selection of the most probable motion model set, to estimate the road section on which 
the target is moving on, is based on Wald’s sequential probability ratio test (SPRT) [9]. 



3.2 Multiple target tracking 

For the MGT problem, we use the SB-MHT (Structured Branching Multiple Hypotheses 
Tracking) presented in [2]. When the new measurements set Z (k) is received, a standard 
gating procedure is applied in order to validate MTI reports to track pairings. The existing 
tracks are updated with VS-IMMC and the extrapolated and confirmed tracks are formed. 
More details can be found in chapter 16 of [2], In order to palliate the association problem, 
we need a probabilistic expression for the evaluation of the track formation hypotheses 
that includes all aspects of the data association problem. It is convenient to use the log- 
likelihood ratio (LLR) L l (k) or a track score of a track T k ’ 1 expressed at current time 
tk- 



4 Target type tracking 

Our approach consists to use the belief on the identification attribute to revise the LLR 
with the posterior pignistic probability on the target type. We recall briefly the Target Type 
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Tracking (TTT) principle and explain how to improve VS-IMMC SB-MHT with target ID 
information. TTT is based on the sequential combination (fusion) of the predicted belief 
of the type of the track with the current “belief measurement” obtained from the target 
classifier decision. The adopted combination rule is the so-called Proportional Conflict 
Redistribution rule no 5 (PCR5) developed in the DSmT (Dezert-Smarandache Theory) 
framework since it deals efficiently with (potentially high) conflicting information. A 
detailed presentation with examples can be found in [4, 7], 



4.1 Principle of the target type tracker 

To estimate the true target type type(k) at time k from the sequence of declarations c(l), 
c( 2), . . . c(k) done by the unreliable classifier up to time k. To build an estimator t.ype(k) 
of type(k), we use the general principle of the Target Type Tracker (TTT) developed in 
[4] which consists in the following steps: 

1. Initialization step (i.e. k = 0). Select the target type frame Crot = {@i, • • • , 9 n } 
and set the prior bba m~(.) as vacuous belief assignment, i.e m~ (9\ U . . . U 9 n ) = 1 
since one has no information about the first observed target type. 

2. Generation of the current bba m 0 bs(-) from the current classifier declaration c(k) 
based on attribute measurement. At this step, one takes m 0 b s {c(k)) = P{c(k)} = 
C c (k)c(k) and all the unassigned mass 1 — m 0 b s {c(k)) is then committed to total 
ignorance 9\ U . . . U 9 n . C c ^)c(k) is the element of the known confusion matrix C 
of the classifier indexed by c(k)c(k). 

3. Combination of current bba m 0 b s (.) with prior bba m _ (.) to get the estimation of 
the current bba m(.). 

4. Estimation of True Target Type is obtained from m(.) by taking the singleton of 
0, i.e. a Target Type, having the maximum of belief (or eventually the maximum 
Pignistic Probability). 

5. Set m~ (.) = m(.); do k = k + 1 and go back to step 2). 

Naturally, in order to revise the LLR in our GMTI-MTT system for taking into account the 
estimation of belief of target ID coming from the Target Type Trackers, we transform the 
resulting bba m(.) = [m~ 0 TO 0 b s ](.) available at each time k into a probability measure. 



4.2 Data attributes in the VS IMMC 

To improve the target tracking process, the introduction of the target type probability is 
done in the likelihood calculation. For this, we consider the measurement z*(fc)(Vj £ 
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{1, . . . , TOfc}) described in (1) and (2). With the assumption that the kinematic and classi- 
fication observations are independant, it is easy to prove that the new combined likelihood 
A l N associated with a track T k ’ 1 is the product of the kinematic likelihood. 



5 Illustration 

In the extended version of this paper, we will illustrate our algorithm by using a complex 
scenario generated with a powerful simulator developed at ONERA. The area of interest 
is located in a Active country called North Atlantis. In this scenario, the goal is to detect 
and track several targets with 2 GMTI sensors (JSTARS, SIDM), 18 UGS and 4 UAV 
(SDTI), in oder to build the situation assessment and evaluate the threat in order to protect 
the coalition forces. On the operation theater, 250 targets evolve, they can maneuver on 
and out the road network. The set of target type is significant, we can have for instance 
civilian vehicles (as 4x4, cars, bus, truck,...) and military vehicles as well (T— 62, AMX 
30, Kamakov,...). llustrations and conclusion of our algorithm will be presented in the 
extended version of this paper. 
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